Sentiment Analysis of IMDb movie reviews

There are hundreds of newspaper articles, blogs, magazines and product reviews that get released on the web everyday. The New York Times has a database of newspapers spanning over 20 years between 1987 and 2007 that is available online. The online database also contains 1.8 million articles from The Times, and many of these online articles have been manually annotated for people, places and organizations. However, an important question is how do we make sense of all this abundant information? The information can possibly be used to infer the popular view of the people on a particular social issue, political situation or even a new movie. Under the assumption that the online sentiment represents the general sentiment of the public, all the abundant information can be used to get an idea of the public sentiment. And if the information is analyzed over a period of time, the change in public sentiment on a particular topic can be tracked over a period of time. The particular type of sentiment analysis problem we address in this work is the problem of sentiment analysis using IMDb reviews. The objective of this work is to draft a procedure that assigns either a positive (1) or a negative (0) sentiment to a given IMDb movie review. So, if we are given k number of IMDb reviews pertaining to a particular movie, we can use our algorithm to assign either a 1 or a 0 to each IMDb review, and then the sentiment assignments to the k movie reviews can be used to get an idea of the popular sentiment of the public towards that movie. A lot of work has already been done on sentiment analysis for movie reviews. The most straight forward method is to count the number of times a word from the model vocabulary appears in a review, and then the count of each word in the model vocabulary is used to form the feature vector for the review. However, this bag of words representation loses the word order so different reviews with identical word composition will have identical vector representation. One of the most prominent model is the Vector Space Model(VSM). In the Vector space model, each word is represented by a vector (known as word vector) and the continuous similarities between words are represented by the distance between their respective word vectors. One of the popular works in unsupervised VSMs is [Maas and Ng 2010] where the context in which a word appears is used to guess the meaning of the word. Thus, words that appear in similar contexts are assumed to be similar in meanings. The semantic similarity between words is captured in the word vector representation of the words such that the word vectors of semantically similar words are close to each other. However, because no sentiment polarity information is associated with the training samples used for learning word vectors, the word vector representation does not capture the sentiment similarity between the words. Thus, the work in [Maas and Ng 2010] was extended in [Maas et al. 2011] which uses the label information of documents to learn vector representation of words that capture semantic as well as sentiment similarities between words. For example, the algorithm in [Maas and Ng 2010] will capture that terrible and aweful are similar in meaning, but it will not capture the negative sentiment associated with these words. However, the word vectors representation for the two words using the algorithm in [Maas et al. 2011] will not only capture the semantic but also the sentiment similarity. These algorithms were further extended in [Le and Mikolov 2014] which proposed a way of incorporating use of unlabeled data with the supervised learning approach proposed in [Maas et al. 2011]. In [Le and Mikolov 2014], each review is assigned a unique paragraph vector. They propose an unsupervised learning framework that learns continuous vector representation for each review where the review vector representation is trained to be useful for learning word vectors. So, when learning a word vector for each word in the model vocabulary, the context in which a word appears is used along with the paragraph vector to learn the word vector representation. So each review has a unique paragraph vector associated with it whereas the word vectors are shared. There are also works that have gone beyond the word-level representation to achieve phrase level representation [Yessenalina and Cardie 2011]. However, in our work, we focus on using wordlevel and paragraph-level representations for sentiment analysis of IMDb reviews. Most of the works on sentiment analysis using IMDb reviews uses the 25k labeled training samples available at http://ai.Stanford.edu/amaas/data/sentiment/index.html for training. The mentioned dataset contains 25k labeled training samples and 50k unlabeled training samples in total. However, we analyze the sentiment analysis problem with the IMDb dataset in the context when labeled training data is not available in abundance. In particular, we suggest a largely unsupervised approach for sentiment analysis that uses the 50k unlabeled training samples for training with a few hundred labeled training samples.